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ABSTRACT  ( 


This  report  contains:  a description  of  the  state  of  affairs  in  global 
optimization  which  existed  at  the  inception  of  this  grant  from  the  user 
point  of  view  and  also  available  algorithms;  the  results  obtained  in  the 
five-year  grant  period  for  (a)  separable  optimization  problems,  (b)  factor- 
able problems,  (c)  infinitely  constrained  problems,  and  (d)  other  specially 
structured  problems;  a brief  summary  of  the  direct  application  of  this  work 
to  Department  of  Defense  planning  studies.  The  basic  result  of  this  research 
is  that  rigorous,  efficient,  implementable  algorithms  are  now  available  for 
obtaining  global  (as  opposed  to  strictly  local)  optimizers  to  nonlinear  pro- 
gramming problems  defined  by  explicit  functional  relationships  and  having 
bounded  variables. 
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1.  ItitrodiictloH 


The  problem  to  which  the  research  in  this  grant  was  addressed  was 
that  of  finding  global  solutions  to  nonlinear  optimization  problems  of 
the  form: 

rain  f (x) 

xee" 

subject  to  the  restrictions  that  gj(x)  > 0 for  i = l,...,m  . For  con- 
vex programming  problems,  local  solutions  to  the  problem  are  global  solu- 
tions and  hence  no  difficulty  exists.  But  for  the  more  general  class  of 
problems  which  are  not-necessarily-convex  (nonconvex)  such  problems  may 
have  local  solutions  which  are  not  global  solutions.  Users  of  nonlinear 
programming  algorithms  are  thus  confronted  with  the  possibility  that  the 
numbers  printed  out  by  computer  coded  algorithms  are  not  the  true  solutions 
to  their  problem.  Most  general  nonlinear  programming  algorltlims  determine 
(at  best)  a local  solution  of  the  given  problem  (l.e.,  a solution  point 
that  Is  optimal  only  in  a neighborhood  or  vicinity  of  the  point) . 

In  1973  when  research  began  on  this  grant,  users  of  nonlinear  pro- 
gramming algorithms  were  usually  advised  to  handle  this  problem  by  start- 
ing from  different  points.  If  the  algorithms  produced  the  same  solution 
point,  then  it  "probably"  was  the  only  local  (and  hence  global)  solution. 

The  state  of  the  art  in  algorithms  which  attempted  to  deal  with  the  local 
vs.  global  problem  was  in  disarray.  In  a survey  of  the  multitude  of  ideas 
for  dealing  with  the  difficulty  (McCormick  [ 1 1)  it  was  found  that  except 
for  a few  papers  (notably  Falk  and  Soland  { 2 ] and  Beale  and  Tomlin  [ 3 )) 
these  ideas  were  unimplementable,  or  were  computationally  unacceptable 
because  of  the  effort  involved.  As  an  example  of  the  former  it  was  sug- 
gested that  the  convex  envelope  of  the  function  f(x)  be  minimized  in  the 
convex  hull  of  the  feasible  region.  The  well-known  "small  grid"  approach  is 
example  of  the  latter. 

The  research  in  the  five-year  period  of  this  grant  has  been  directed 
toward  developing  efficient,  impleroentable,  rigorous  algorithms  for  guar- 
anteeing the  attaining  of  global  solutions  to  nonconvex  programming  prob- 
lems. This  research  has  been  successful  in  several  areas.  In  Section 
2 is  a summary  of  the  research  on  separable  programming  problems.  Section  3 
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conCains  the  results  obtained  on  the  very  general  class  of  factorable 
programming  problems.  Section  4 discusses  infinitely  constrained  pro- 
blems and  in  Section  5 are  results  for  other  structured  problems.  Some 
of  the  work  has  been  applied  to  Department  of  Defense  planning  models, 
and  this  is  taken  up  briefly  in  Section  6.  Work  remaining  to  be  done, 
mostly  in  the  area  of  implicitly  defined  problems  is  described  in 
Section  7. 

Papers  supported  by  the  grant  are  indicated  with  an  asterisk  in  the 
reference  section  at  the  end. 

2.  Separable  Programming 

The  Palk-Soland  algorithm  for  separable,  non-convex  programs  ( 2 ) 
was  later  extended  by  Soland  [ A ] to  apply  the  underlying  branch  and 
bound  methodology  directly  to  each  of  the  functions  defining  the  con- 
straints. A parallel  effort  was  carried  out  by  Falk  [ 5 ] who  addressed 
piecewise  linear  separable  problems,  and  effectively  handled  the  con- 
straints via  a Lagranglan,  or  dual  problem.  The  latter  approach  was 
shown  by  Greenberg  [ 6 ] to  yield  sharper  bounds  than  any  method  which 
took  convex  envelopes  of  the  constraining  function  individually.  In  a 
later  paper,  Falk  [ 7 ] extended  Greenberg's  results,  and  characterized 
problems  wherein  the  bounds  obtained  were  Identical. 

The  work  on  the  piecewise  linear  problems  Ims  proven  to  be  efficient 
and  practical,  and  a number  of  the  applications  addressed  later  were 
effectively  solved  via  this  method.  Hoffman  [ H ] coded  a reliable 
version  of  the  method  during  the  contract  period,  which  was  later  re- 
programmed and  extended  by  Grotto  f ) of  TDA. 

Al-Khayyal  [10]  has  obtained  an  efficient  extension  of  the  original 
Falk-Soland  method  as  applied  to  "Bi-Convex  Programs."  This  problem  may 
be  considered  as  more  general  than  a separable  problem,  and  a .special  case 
of  the  factorable  problem  addressed  in  Section  3. 

Recent  developments  beyond  the  contract  period  are  being  m.ade  in 
the  solution  of  separable  equilibrium  problems,  and  in  multi-level 
optimization  problems. 
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3.  Factorable  ProRrammlnt; 

Almost  all  functions  used  In  nonlinear  programming  for  practical 
problems  are  factorable  functions,  i.e.,  functions  of  n variables 
which  arc  complicated  compositions  of  transformed  sums  and  products  of 
transformations  of  transformed  sums  and  products  of  functions  of  a single 
variable,  branch  and  bound  algorithms  for  obtaining  global  solutions 
usually  require  that  convex  underestimating  problems  be  available  for 
non-convex  problems  restricted  to  hyperrectangles.  As  was  mentioned  in 
the  previous  section,  this  is  fairly  easy  to  do  for  separable  problems. 

In  McCormick  1976  [11]  is  a procedure  for  doing  this  when  the  problem 
functions  are  factorable.  This  paper  also  contains  splitting  rules  to 
use  in  a branch  and  bound  scheme  for  obtaining  global  solutions.  Con- 
vergence to  a global  optimizer  was  proved. 

In  addition  to  the  use  of  convex  underestimating  problems,  another 
tool  in  algorithms  for  non-convex  programming  is  the  ability  to  verify 
that  a local  minlmizer  is  a global  mlnlmizer  in  a convex  compact  set. 

For  twice  continuously  differentiable  functions  minimized  subject  to 
bounded  variables,  the  theory  for  verifying  that  a local  unconstrained 
minlmizer  existed  in  a hyperrectangle  of  bounds  and  placing  tight  bounds 
on  its  location  was  given  in  Manclni  and  McCormick  [12].  A paper  develop- 
ing Che  practical  computational  aspects  of  this  theory  was  also  done 
(Manclni  and  McCormick  [13])  and  will  be  puhlislved  in  Opel  at  Ions  Researcii 
in  1979.  This  was  illustrated  by  an  example  of  the  optimal  design  of 
a refinery. 

The  generalization  of  this  verification  principle  for  general 
non-convex  programs  was  Just  completed  and  the  paper  has  been  submitted 
to  Math,  of  O.R.  [14]. 

These  four  papers  complete  the  requirements  for  an  algorithm  to 
obtain  global  solutions  to  factorable  programming  problems  with  bounded 
variables. 
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4 . Infinitely  Coustraluod  Problems 

A preliminary  study  done  by  Bracken  and  McGill  [15]  involving 
optimization  problems  with  optimization  problems  defining  the  constraints 
led  to  a formulation  of  a general  mathematical  program  with  an  infinite 
number  of  constraints.  An  effective  solution  procedure,  based  on  a re- 
laxation of  the  constraints,  was  developed  and  published  by  Blankenship 
and  Falk  [16]. 

The  basic  algorithm  involves  the  solution  of  a sequence  of  finitely 
constrained  problems,  with  a check  for  optimality  at  each  step.  The  check 
itself  may  involve  the  solution  of  an  optimization  problem  (as  in  the 
Bracken  and  McGill  application),  or  may  be  trivial  (as  in  certain  schedul- 
ing problems,  e.g.,  [17]). 

Convergence  was  established  under  very  general  conditions,  and  hence 
applies  to  non-convex,  as  well  as  convex  problems. 

The  general  context  of  the  problem  and  its  solution  procedure  has 
lent  itself  to  a wide  variety  of  applications  (well  beyond  the  Bracken- 
McGill  model),  some  of  which  are  mentioned  in  Section  7.  Certain  varieties 
of  "raax-min"  problems  are  especially  well  treated  with  tills  method.  And 
the  coupling  of  this  method  with  the  non-convex  methodology  developed  here 
at  The  George  Washington  University  has  resulted  in  a practical  tool  that 
can  be  applied  to  previously  unsolvable  problems  (see,  e.g.,  [18]). 

5.  Algorithms  for  Structured  Non-separable  Troblems 

The  principal  difficulty  in  non-convex  programming  stems  from  the 
necessity  of  compiling  global  estimates  of  a function's  behavior  based 
on  relatively  few  point  calculations.  Separability  alleviates  the 
situation  considerably,  since  it,  in  effect,  allows  one  to  treat  problems 
in  several  variables  by  using  function  values  of  several  functions  of 
single  variables.  If  these  functions  I>ave  special  structure  (e.g.,  con- 
vex, or  concave),  the  Falk-Soland  algorithm  may  be  directly  employed. 
Otherwise,  piecewise  linear  approximations  may  be  employed  as  in  [5  ]> 

^ The  problems  wherein  separability  is  difficult  or  expensive  to 

obtain,  only  specially  structured  problems  may  be  treated.  One  such 
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problcm  that  was  solved  under  the  grant  was  the  problem  of  minimizing 
a concave  function  over  a linear  polyhedron.  Falk  and  Hoffman  (l9l 
wore  able  to  compute  convex  envelopes  of  the  objective  function  over 
successively  tighter  relaxations  of  the  constraint  region,  thereby 
forcing  convergence  when  the  relaxations  agreed  with  the  feasible 
region.  The  relaxations  were  tightened  by  adding  original  constraints 
successively,  so  that  finite  convergence  was  guaranteed. 

Several  applications,  including  a selective  strike  model  for  bomber 
deployment  are  discussed  In  Hoffman  [20]- 

Another  class  of  problems  which  could  be  separated,  at  t!ie  expense 
of  additional  variables  and  constraints,  was  treated  directly  by  Al-Khayyal 
[lO].  These  problems  are  characterized  by  the  presence  of  terms  of  the 
form  x^y^  , and  appear  to  be  linear  in  a set  of  variables  If  all  other 

variables  are  held  constant,  and  conversely.  The  basic  algorithm  of 
Falk-Soland  [ 2 ] was  generalized,  with  appropriate  modifications,  and 
convergence  proofs  obtained. 

Immediate  applications  to  "linear  complementarity"  i^robloms  were 
obtained. 

Filially,  an  algorithm  for  producing  global  solutions  to  "signomial" 
programs"  was  first  published  in  [21J. 

6 . Application  of  Non-convex  Programming 

In  addition  to  an  unusually  high  number  of  methodological  results, 
this  contract  has  also  produced  an  exceptional  list  of  direct  applications 
of  the  theory. 

A tool  (TAG  CONTF.NOER)  for  assessing  figures  of  merit  for  develop- 
mental aircraft  was  generated  by  Air  Force  personnel,  and  used  to  support 
policy  decisions.  There  were  at  least  two  major  flaws  in  the  model,  one 
resulting  from  convergence  to  a local,  as  opposed  to  global,  solution. 

These  are  discussed  in  [22]. 

A weapons  allocation  model  first  developed  by  Shore  and  Wingate 
[23]  was  significantly  extended  and  solved  by  Falk  and  Huffman  [18]. 
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Methodology  relating  to  non-convex  programming  as  well  as  the  Infinitely 
constrained  work  was  used  to  generate  solutions.  It  Is  Important  to 
note  that  a model  of  this  type  of  complexity  could  not  have  been  solved 
with  the  tools  available  before  the  grant  period. 

An  alternative  to  the  aforementioned  TAG  CONTENDER  was  published 
by  Bracken,  Falk,  and  Karr  [24].  While,  as  opposed  to  TAG  CONTENDER, 
the  new  model  solved  sequential  games  exactly.  It  could  only  apply  to 
games  of  relatively  short  duration.  The  philosophy  of  infinitely  con- 
strained programming  was  used  to  accelerate  convergence. 

Additional  comments  on  the  multi-stage  games  are  contained  In  the 
IDA  paper  [25], 

Another  application  of  the  combined  non-convex/infiniteiy  con- 
strained methodology  was  obtained  In  a project  scheduling  context  [17] 
and  [26],  Here,  the  problem  can  be  cast  in  our  optimization  format, 
but  with  an  excessively  large  number  of  constraints  (the  number  of 
variables,  factorial).  Infinitely  constrained  programming  was  shown 
to  be  capable  of  generating  solutions  to  an  apparently  unsolvable  pro- 
blem. Non-convex  cost  functions  could  easily  be  handled. 

A two-strike  nuclear  exchange  model  was  first  formulated  and  solved 
by  Bracken,  Falk  and  Miercourt  [27].  The  intrinsic  non-convex  nature 
of  the  problem  was  easily  handled  by  techniques  developed  in  this  study. 

7 . Summary 

In  Sections  1-6  the  main  results  are  listed  in  detail.  Section  8 
contains  the  references  for  those  sections.  The  papers  credited  to  the 
Grant  are  denoted  with  an  asterisk  (*) . Other  byproducts  of  the  research 
sponsored  by  the  Grant  are  listed  in  Section  9.  These  also  are  contained 
preliminary  reports  eventually  published  in  the  open  literature  and 
referenced  in  Section  8. 

Work  remains  to  be  done  in  the  area  of  implicitly  defined  problems. 

An  unusually  large  body  of  theoretical  results  of  an  applicable 
nature  were  generated  by  this  grant.  In  addition  to  effectively  build- 
ing a workable  set  of  algorithms  for  the  non-convex  problem,  other 
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spin-offs  were  found  In  the  fields  of  infinitely  constrained  programming, 
and  in  multi-stage  games. 


The  theoretical  results  obtained  in  this  study  were  directly  applied 
to  a wide  variety  of  models  with  applications  of  specific  interest  to 
the  Air  Force. 
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